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Abstract 



Alpha-Max is a multiple comparison procedure which is based on the Bonferroni 
inequality. However, rather than using pre-set adjusted alpha values for each pairwise 
contrast. Alpha-Max bases the decisions on the actual p-values for the pairwise contrasts. 
After a significant omnibus F test, the pairwise p-values are determined and ordered 
from lowest to highest. Alpha-Max includes as being significant only those contrasts 
whose sum of sequential additive p-values is less than the nominal alpha desired. 

Results for Alpha-Max are compared with results using the original Bonferroni, Tukey’s 
HSD, and Student-Newman-Keuls (SNK) procedures based on simulated three and four- 
group data sets with varying mean patterns and sample sizes. In every case, Alpha-Max 
had more power to detect pairwise differences as compared with Bonferroni and HSD 
and in one mean pattern was more powerful than SNK in detecting mean differences at 
r= K-l steps. Needed additional research and possible applications of this procedure are 

discussed. 
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Alpha-Max: A Novel New Multiple Comparison Procedure 



Over the past several decades, methods of comparing means following a 
significant omnibus F test have been proposed and used to make decisions regarding 
pairwise and more complex comparisons. These methods have involved different 
philosophies on controlling Type I error. They have included methods where the Type I 
error rate control has been based on each individual comparison, referred to as 
hypothesis-wise or comparison-wise control (e.g., Fisher's Least Significant Difference 
Procedure or LSD) and methods where Type I error rate control has been based on an 
entire set of comparisons, referred to as experiment-wise or family-wise control (e.g., 
Tukey's Honestly Significant Difference Procedure or HSD, Bonferroni, and Scheffe). 
Other methods have taken stepwise approaches to controlling Type I error (e.g., Student- 
Newman-Keuls Procedure or SNK, Duncan’s new multiple range, and variations of the 
Bonferroni approach). Descriptions and computational information on all of these 
procedures, except the variations of the Bonferroni approach, can be found in most any 
statistics book that includes the analysis of variance (e.g., Kirk, 1982). 

In general, the most commonly used post hoc approaches have been Tukey’s 
HSD, the Bonferroni, and the Student-Newman-Keuls. Each of these methods have 
their own advantages and disadvantages. While Tukey’s HSD controls for experiment- 
wise Type I error, it is considered to be somewhat conservative. The Bonferroni, when 



used for ail pair-wise comparison, is also conservative, even more conservative than the 
HSD. The SNK is considered to have the highest power of the three, but it does not 
control the experiment-wise Type I error. 

Because of the lack of computing availability to researchers when these methods 
were developed, tables of critical values were prepared to aide in their use. However, 
now that computing power is available on a mass scale, it is possible to consider the 
actual probabilities or p-values of outcomes in making decisions. Alpha-Max is a 
procedure that uses the actual Type I probabilities or p-values to make decisions about 
which pairwise differences are to be considered statistically significant. The rationale is 
to order the pairwise difference p-values from lowest to highest, moving from the lowest 
to the highest, accumulating the actual probabilities until the addition of the next higher 
probability exceeds the pre-set nominal alpha level. All pairwise differences whose p- 
values are already included in the set are considered significant, but the one(s) which 
result in the sum of the probabilities being higher than the a priori alpha is/are not 
considered to be significant. A similar procedure referred to as the sequentially rejective 
Bonferroni test was proposed by Holm (1979). However, in Holm’s procedure, each 
comparison alpha for sequential test was based on a new alpha value determined as a 
function of the experiment-wise alpha and the step in the sequence. With the removal of 
this restriction, Alpha-Max increases its power in the first comparisons. The effect of 
this on experiment-wise Type I error is yet to be assessed, although we believe it will be 
in an acceptable range as compared with other competing methods. 

In this research, simulated data sets are used to provide examples of the outcomes 
associated with this procedure compared with Bonferroni, HSD, and SNK approaches. 
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The purpose is to compare results from Alpha-Max with the three other approaches. It 
is predicted that Alpha-Max has higher power to detect differences as compared with 
Bonferroni and HSD, but less power to detect differences as compared with SNK. 
However, AJpha-Max does control the experiment-wise Type I error which SNK does 
not. If the sum of actual Type I probabilities in Alpha-Max is less than alpha, then Type 
I error cannot be higher than alpha. 

Relevant Literature on Multiple Comparison Procedures 

The literature on multiple comparison procedures can be divided into two primary 
categories: articles that propose multiple comparison procedures and articles that 
compare multiple comparison procedures. This review will only present literature that 
addresses Bonferroni procedures or compares multiple comparison procedures as this 
paper is primarily concerned with comparing Alpha-Max with the HSD, Bonferroni, and 
SNK. Both analytical and simulation methods are common techniques for comparing 
methods. However, most recent comparisons have been done using simulations. The 
most common dependent variables in the comparison have been Type I error or its 
probability, alpha, and power. This section reviews error rates and comparing multiple 
comparison procedures. 

Error Rate 

The most common criteria for judging multiple comparison procedures is the 
Type I error rate. Essentially, error rates can be defined as experiment-wise, hypothesis- 
wise, or somewhere in between. The error rates are related in the following manner 



when the individual tests are orthogonal: 

a EW = 1 “ (1 “ a H\v) C 

where EW = experiment-wise, HW = hypothesis-wise, and C = number of comparisons 
in the experiment (Ottenbacher. 1991; Toothaker, 1993). In general, the relationship can 
be written as an inequality also covering the cases where the individual comparisons are 
not orthogonal: 

a EW — 1 - (1 - a Hw) C 

This inequality and the Bonferroni inequality (Hayes, 1988) are the basis for setting the 
error rates for most of the multiple comparison procedures. 

Comparing Multiple Comparison Procedures 

A number of studies have compared the ability of multiple comparison procedures 
to find significant differences (e.g., Klockars & Hancock, 1994; McCarroll, Crays, & 
Dunlap, 1992; McFatter & Gollob, 1986; Myette & White, 1982, March; Olejnik & Lee, 
1990, April; Seaman, et al., 1989, March; Toothaker, 1993; Zwick, 1991). Results of 
these studies have been very consistent and suggest that procedures that control the 
experiment-wise error are less powerful than those that do not. Procedures that control 
only hypothesis-wise error rates tend to be the most powerful and procedures that use 
step-down procedures somewhere in between. Applying this to the three procedures 
examined in this study, it results in the Bonferroni being the least powerful when applied 
to all pairs (experiment-wise error rate), the Tukey HSD being more powerful (also 
experiment-wise error rate), and the SNK being the most powerful (step-down 
procedure), but with lowest control of Type I error.. 



Theoretical Rationale for Alpha-Max 

The logic for Alpha-Max is rather simple. Arrange the hypotheses by p-values 
from the smallest to largest and accumulate the p-values until they exceed the nominal 
alpha. All of the hypotheses prior to the one associated with the first nonsignificant p- 
value are significant. Alpha can be thought of as money you have to spend on power. 

The most powerful way to spend this money is on comparisons that do not cost very 
much (i.e., have small p-values). Thus, ordering the hypotheses based on p-values from 
smallest to largest and buying (testing) them in that order would get you the most 
information for the least cost. 

The theoretical rationale for Alpha-Max is equally simple. It is based on the 
Bonferroni inequality (Hayes, 1988, p. 411). 

Alpha-Max and the Bonferroni Inequality 

Simply stated, the Bonferroni inequality establishes that the experiment-wise error 
rate is less than or equal to the sum of the errors for the individual tests making up the 
experiment. Symbolically, the Bonferroni inequality is as follows: 

®EW — Qq + dfj ••• "b 

Thus, by selecting only those tests that have error rates that sum to less than or equal to 
the desired experiment-wise error rate, control of the experiment-wise error rate at the 
desired level is assured. 

This is the same rationale used with the common Bonferroni procedure except 
that Alpha-Max does not require one of the restrictions. The Bonferroni procedure was 
established by requiring that each individual test be run at the same level of significance. 



Thus, a„ a,, through a c were assumed to be equal to a HW , the hypothesis-wise error. In 
this case, the Bonferroni inequality becomes: 

a EW — ^ a HW 

Since each hypothesis-wise a is a constant, the inequality becomes: 

a EW — C**HW 

where C is the number of individual tests that comprise the experiment. By dividing 
both sides of the inequality by C, it is clear that by choosing the hypothesis-wise alpha 
equal to the experiment-wise alpha divided by the number of tests (a H w = a Ew / C)» the 
desired experiment-wise alpha will be maintained. 

Alpha-Max accomplishes this same result by selecting only those individual 
comparison alphas whose sum is less than or equal to the desired experiment-wise alpha. 
We have proposed an additional level of protection by requiring that Alpha-Max be 
preceded by a significant omnibus F test as required by Fisher when he proposed the 
LSD procedure. 

It should be noted that the Bonferroni inequality would be an equality if the 
individual comparisons were independent. This is the basis for multiple comparison 
procedures such as the orthogonal comparison procedure and Dunnett's procedure. In 
cases where the individual comparisons are not independent, the Bonferroni test 
becomes more conservative as will the Alpha-Max procedure. Thus, based on this logic 
and the Bonferroni inequality, we can be assured that the experiment-wise alpha will be 
less than or equal to the sum of the p-values for the selected tests. If the individual tests 
are selected such that the sum is less than or equal to our desired experiment-wise alpha, 
than the experiment-wise error rate will be controlled. For example, if we choose the 
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experiment-wise error rate to be .05 and select the individual comparisons such that 
.05 < a, + a 2 + ... + a c , than the experiment-wise error rate will be less than or equal 
to .05. 

Other Bonferroni-Type Procedures 

There have been many adaptations of the basic Bonferroni procedure over the 
years. Most of these modified the application of the Bonferroni inequality to increase 
the power of the comparisons. The simplest modification was to a priori reduce the 
number of comparison being tested. Most of the modifications involved some type of 
sequential application of the Bonferroni inequality (e.g., Hochberg, 1988; Holland & 
Copenhaver, 1987; Holm, 1979; Hommel, 1988; Li, Olejnik, & Huberty, 1992, April; and 
Rom, 1990). Each of these modifications involved ordering the pair-wise comparisons 
based on the p-values and applying a modified alpha-value in a sequential manner. A 
number of articles reported comparisons among these and other multiple comparison 
procedures (e.g., Supattathum, Olejnik, & Li, 1994, April; de Cani, 1984; Edwards, 1991, 
April; Li, Olejnik, & Huberty, 1992, April; Ottenbacher, 1991). The findings tend to be 
that the original Bonferroni procedure is least powerful with the modified versions 
having varying degrees of power. However, Supattathum, Olejnik, & Li (1994, April) 
concluded that the increase in power over the original Bonferroni approach was not 
particulary high, especially when nominal alpha was .05.. 

Based on these studies, the Alpha-Max procedure will have as much or more 
power than the other procedures that are variations of the Bonferroni procedure. There 
may be, however, greater Type I error risk. This greater Type I error risk is due to the 
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non-random nature of the application of the test criteria. It is similar to using an a 
priori test criteria but not determining which groups to compare until after examining the 
differences. Only a Monte Carlo type simulation study could determine this with any 
degree of certainty. 



Research Design and Analysis 

Data Sets 

Two three-group and three four-group data patterns were specified in accordance 
with Cohen’s (1988) description of group mean patterns. Each of these was used with 
three group sizes, n= 10, n= 30, and n= 100. Each data set was normally distributed 
and within each sample size configuration, group variances were equal. 

Group Mean Patterns 

Pattern 1 has one group with a low mean, one with a high mean, and all other 
groups with means exactly between the low and high means. This pattern has the lowest 
variance of the group means. Pattern 2 has all groups with means equally distant from 
each other. This has medium group mean variability. When there are three groups, 
patterns 1 and 2 are the same. Pattern 3 has half of the means at the low point and half 
of the means at the high point, a pattern with the highest group mean variability. When 
there is an odd number of groups, pattern 3 does not have an equal number of group 
means at the low and high points. There would be K/2-.5 at one end and K/2+.5 at the 

other. 
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Analysis 

Within each group number, group size, and pattern configuration, the range of 
lowest and highest means was increased until a significant omnibus F statistic, at p<.05, 
was reached. In addition to the omnibus F statistic, all pairwise comparisons were tested 
using the original Bonferroni approach, Tukey’s HSD, Student-Newman-Keuls, and 
Alpha-Max. The range was systematically increased by small intervals until every 
possible difference was detected by all four multiple comparison methods. The order in 
which the differences were found by each method at each step was determined and is 
reported in these results. All of these analyses were conducted using SPSSx, mainframe 
software at the University of Alabama. Alpha-Max pairwise tests were determined using 
weighted contrast statements and were based on pooled variance t probabilities. In 
addition, power was determined using the methods presented by Cohen (1988) which 
take into account the different patterns and omega-square was determined as the 
measure of strength of the among group variance. 

Results 

Three Group Arrangements 

Pattern 1-2 . In the three group situation, patterns 1 and 2 are the same. There is 
one low group, one in the middle, and one high group. Pattern 1-2 results for each of 
the sample sizes are presented in Tables 1-3. For n= 10 (Table 1), the omnibus F 
statistic was significant at 1.20o. At this point, the largest contrast Ov/^) was significant 
for all four methods. At 1.90o, the SNK found the remaining two equally distant, 
pairwise differences (^i, l -/x 2 and /x 2 -/x 3 ) significant; at 2.15 cr, Alpha-Max found these 




12 



9 



differences significant; at 2.25cr, the HSD found them different; and at 2.30 a, the 
Bonferroni found them significantly different. 

Similar results were found for the n= 30 situation (Table 2). The omnibus F was 
significant at 0.65cr as was the /x r /i 3 difference for all four methods. The remaining 
pairwise differences were significant using SNK at 1.05o, using Alpha-Max at 1.20o, using 
HSD at 1.25cr, and at 1.30o for the Bonferroni. For the n= 100 situation (Table 3), the 
omnibus F was significant at 0.35o, where the /x r /x 3 contrast was found significant by all 
four methods. The remaining pairwise differences were significant using SNK at 0.575a, 
using Alpha-Max at 0.65o, using HSD at 0.675a. and using Bonferroni at 0.70 a. 

Pattern 3 . Results for Pattern 3, all means at the extremes, are presented in 
Tables 4-6. In this case, the means for groups 1 and 2 were the same at the low end 
compared with the mean of group 3 at the high end of the range. In this pattern, there 
are only two possible differences, and As such, the highest difference is the 

only difference and it occurs twice. Thus, there are two differences tied at r= 3 steps for 
the SNK. Because of random variation, this is unlikely to actually occur in real data 
situations. For this analysis, they will both be treated as being significantly different at 
the highest number of steps. 

For the n= 10 situation (Table 4), the omnibus F was significant at l.Olo. At this 
point, both differences were considered significant using the SNK. At 1.07 a, Alpha-Max 
found the differences significant, they were found different at l.llo using HSD, and at 
1.15a using Bonferroni. For the n= 30 situation (Table 5), results were very similar. 

The omnibus F was significant and both pairwise differences significant using SNK at 
0.56o. At 0.59 a, Alpha-Max found both significant; at 0.61a, HSD found them 



significant; and at 0.64o, the Bonferroni found the differences significant. For n 100 
situation (Table 6), the omnibus F and SNK differences were found at 0.305a. At 0.32a, 
AJpha-Max found the differences; at 0.335a, differences were determined by HSD; and at 
0.345a, differences were found using the Bonferroni approach. 

In every three-group pattern situation, across all three sample sizes, SNK detected 
significant differences first, Alpha-Max detected significant differences second, followed 
by HSD third, and lastly by Bonferroni. 

Four Group Arrangements 

Pattern 1 . Four-group, pattern 1 mean comparisons are reported in Tables 7-9. 

In this case the largest mean difference is the r= 4 step case, with the contrast of av/V 
Since the two means in the middle and ^ ) are the same, the contrast is zero. 

The other four contrasts are equal in mean difference. These are: tirth, fh’f 1 4> an< * 

Table 7 presents the results for the n= 10 situation. At 1.35a, the omnibus F was 
significant and the /VA 4 difference was found to be significant by all four methods. At 
1.90a, SNK found the other four mean differences significant. At 2.36a, Alpha-Max 
found these differences significant; at 2.44a, HSD found the differences significant; and 
at 2.50a, Bonferroni found the differences significant. 

The same orders of difference were found with the n= 30 and n= 100 situations. 
In the n= 30 situation (Table 8), the omnibus F and the difference was found at 
0.75a. The other four pairwise differences were found by SNK at 1.05a, by Alpha-Max 
at 1.32a, by HSD at 1.35a, and by Bonferroni at 1.40a. In the n= 100 situation (Table 
9), the omnibus F and the /v/a 4 difference was found at 0.40a. The other four pairwise 



differences were found by SNK at 0.575a, by Alpha-Max at 0.71a, by HSD at 0.74a, and 
by Bonferroni at 0.75a. In all three, four-group, pattern 1 sample size situations, SNK 
found the differences first, Alpha-Max was second, followed by the HSD and Bonferroni. 

Pattern 2 . In the four group, pattern 2 situation the means are equally spaced 
across the range of means. There is one r= 4 step mean difference which is represented 
by the contrast. There are two r= 3 step mean contrasts and /i. 2 -/i. 4 ) and 

three r= 2 step mean contrasts 2 , /x 2 -/j- 3 , and ^i 3 -ja 4 ). Results for the three sample 
size configurations are presented in Tables 10-12. In the n= 10 situation (Table 10), the 
omnibus F was significant at 1.30a. AH four methods found the £i[-/i. 4 contrast significant 
at this point. The two r= 3 step differences were found significant first by Alpha-Max at 
1.60a, followed by SNK at 1.70a. HSD found the r= 3 contrasts significant at 1.85a and 
Bonferroni found them significant at 1.90a. Relative to the three r= 2 contrasts, SNK 
found them significant at 2.80a, Alpha-Max found them significant at 3.40a, HSD found 
them significant at 3.70a, and Bonferroni found them significant at 3.80a. 

For the n= 30 situation (Table 11) the omnibus F and the difference were 
found significant by all four methods at 0.70 a. The two r= 3 step means were found 
significant at 0.90a by Alpha-Max. SNK found these differences significant at 0.95a, 

HSD found them significant at 1.02a, and Bonferroni found them significant at 1.05a. 
The remaining three pairwise differences were found significant by SNK at 1.55a, by 
Alpha-Max at 1.90a, by HSD at 2.05a, and by Bonferroni at 2.10a. Very similar results 
were observed for the n= 100 situation (Table 12). The omnibus F and the r= 4 step 
contrast were significant at 0.40a. The two r= 3 step mean differences were found 
significant at 0.50a by Alpha-Max. SNK found these differences significant at 0.52a, 
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HSD found them significant at 0.55o, and Bonferroni found them significant at O. 6 O 0 . 

The remaining three pairwise differences were found significant by SNK at 0.85o, by 
Alpha-Max at 1.05o, by HSD at l.lOo, and by Bonferroni at 1.15o. 

In all three four group, pattern 2 situations, Alpha-Max detected the r= 3 step 
mean differences before all other methods including SNK. However, at the r= 2 step 
mean differences the order of Alpha-Max and SNK detection were reversed. Both 
Alpha-Max and SNK detected all differences before HSD and Bonferroni. It may be 
that this is a group mean configuration where Alpha-Max is clearly preferable to all of 
the other methods in detecting the mean differences at r= K-l steps, and perhaps at 

other higher or lower steps when K is large. 

Pattern 3 . In the four group, pattern 3 situation two means are tied at the low 
mean value and two tied at the high mean value. Results for comparing all pairwise 
differences for the three sample size configurations having this pattern are found in 
Tables 13-15. With this pattern, there are four mean difference contrasts that are tied at 
the same difference: \hr\h, #**■#*»• Due to random variation, in a typical real 

data situation, this situation is not likely to happen. 

In the n= 10 situation, presented in Table 13, the omnibus F was found 
significant at 0.95o. At this point the tied four contrasts were found different with SNK. 
Alpha-Max found these four contrasts significant at 1.20a, HSD found them significant at 
1.22a, and Bonferroni found them significant at 1.25a. Results for the n= 30 and n = 

100 situations were very similar. As indicated in Table 14, the omnibus F and the SNK 
tied differences were found significant at 0.53a. Alpha-Max found the four pairwise 
contrasts significant at 0.66a, HSD found the differences at 0.68a, and Bonferroni found 



the differences significant at 0.70a. Table 15 presents results for the n= 100 situation. 
The omnibus F and the SNK tied differences were found significant at 0.29a. Alpha- 
Max found the four pairwise contrasts significant at 0.36a, HSD found the differences 
significant at 0.37a, and Bonferroni found the differences significant at 0.38a. 

While it appears the SNK is much more powerful in this case, that would not be 
the case if the means were not tied. Actually, one of these mean differences would be at 
the i= 2 level. The Alpha-Max, HSD, and Bonferroni methods found differences at 
about the same range value. However, if the means were not tied, Alpha-Max would 
likely have detected differences at much lower range values as compared to HSD and 
Bonferroni. 



Conclusions 

In every case studied, Alpha-Max was more powerful than HSD and Bonferroni. 
For patterns 1 and 2, all methods detected the largest difference at the same point as the 
omnibus F statistic being determined as significant. In the pattern 2 case, at the K-l 
steps, Alpha-Max is more powerful than all of the methods compared including SNK. At 
the r= 2 step situation, SNK is the most powerful, followed by Alpha-Max, HSD, and 
Bonferroni. However, at this level SNK is recognized as not having desirable control of 
Type I error rate. In the perfect status of the pattern 3 situation, SNK is clearly the 
most powerful and the other three methods are very close relative to power. Random 
variations around this pattern are likely to show Alpha-Max to have clearly higher power 
than the HSD and Bonferroni. 

While these results indicate that Alpha-Max has potential of providing a more 





powerful alternative to HSD and Bonferroni in every case and higher than SNK in the 
pattern 2 case and a method with more desirable control of Type I error as compared 
with SNK, there is another very strong advantage to Alpha-Max. It is highly flexible. 

The only requirement for its use is being able to determine the contrast probabilities, 
which is easily done with current microcomputer and mainframe software. Thus, it could 
be used for nonparametric as well as parametric multiple comparisons. It could be used 
with nonorthogonal, planned comparisons as an alternative to the Bonferroni. It has 
promise for being able to test contrasts in the lack of homogeneity of variance situation 



A promising application of the Alpha-Max Procedure is as an a priori multiple 
comparison procedure. To use it is this way, the first step is to put the hypotheses in 
priority order. After the p-value associated with each hypothesis is obtained, begin 
accumulating the p-values as before. All hypotheses are significant as long as the sum of 
the accumulated p-values is less than or equal to the desired experiment-wise alpha. 
Applying Alpha-Max in this way will eliminate the possibility of a slightly inflated 
experiment-wise error rate. It also is in keeping with the advice of many researchers 
(e.g., Thompson, 1990, April; Wang, 1993, November) who recommend using a priori 
multiple comparisons. 



At this point, we have proposed Alpha-Max as a new approach to conducting 
multiple comparisons. While the use of simulated group mean patterns with varying 
sample sizes has demonstrated that Alpha-Max is at least as powerful as Tuky’s HSD and 



as well. 



Future Research 
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Bonierroni, and in some cases more powerful than SNK, there is still a need to conduct 
more extensive research on this method. The primary need is to empirically establish, 
through use of Monte Carlo studies, actual Type I error rates in the various applications 
of Alpha-Max. Another need is to compare Alpha-Max results with those from the 
modifications of the Bonferroni procedure. If these studies conclude that Alpha-Max has 
as good or better control of Type I error, then applications should be examined. 

Future studies should examine applications of Alpha-Max in situations of: lack of 
homogeneity where p-values based on separate variance could be used with Alpha-Max; 
its use as a nonparametric multiple comparison procedure: its use in nonorthogonal. 
planned procedures; its use with directional tests (even different directions on some 
contrasts compared with others) in orthogonal or nonorthogonal planned comparisons; 
its use as an alternative to Dunnett’s tests for comparing groups with a predetermined 
control group; and its use in followup of interaction cell means. 
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Table 1 



Significant Pairwise Differences for Four P ost-hoc Methods, 
K= 3. Pattern 1-2, n= 10. a= 0 05 





Range 


1.20<r 


1.90a 


2. 15<r 


2.25a 


2.30a 




F 


3.60 


9.03 


11.56 


12.66 


13.23 




Pr> F 


.0411 


.0010 


.0002 


.0001 


.0001 




Power 


.62 


.96 


.99 


.99 + 


.99 + 




OT 


.148 


.349 


• 413 


.437 


.449 


Procedure 


Contrast ffpIPjP 












Mt“M3 


* 


* 




* 




Bonferroni 


Mr/^2 






































Tukey’s 


MrM3 


* 


♦ 


★ 


111 1|| 


* 


HSD 


MrMs 








§llg§l 


* 


















M2-M3 












Alpha-Max 


M1-M3 


.012* 


.000* 


Mi 


. 000 * 


.000* 


with Type I 


Mt “M2 


.191 


.043 


; fi23*' 


.018* 


.016* 


Probabilities 


M2“M3 


.191 


.043 


IlMlI 


.018* 


.016* 


Student 


MrM 3 


* 




+ 


* 


* 


Newman- 


M1-M2 






* 


* 


* 


Keuls 


M2“M3 






* 


* 


* 



* Significant contrast at g< .05. 
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Table 2 



Significant Pairwise Differences for Four Post-hoc Methods, 
K= 3. Pattern 1-2. n= 30. a= 0.05 





Range 


0.65(j 


1.05cr 


1.20(t 


1.25(7 


1.30(t 




F 


3.17 


8.27 


10.80 


11.72 


12.68 




Pr>F 


.0470 


.0005 


.0001 


.0000 


.0000 




Power 


.60 


.95 


.98 


.99 


.99 + 




or 


.046 


.139 


.179 


.192 


.206 


Procedure 


Contrast 
















MfMs 


* 




4 


4 


Hi 




Bonferroni 


MrM? 










lillli 






MrM 3 














Tukey’s 


Mt‘M 3 


4 


♦ 


4 


«/V. 


4 




HSD 


Mi 










4 


! 




MrM 3 










4 


1 


Alpha-Max 


MrM 3 


.014+ 


.000+ 




. 000 + 


.000* 


with Type I 


MrM? 


.211 


.045 


Hi 


.018* 


.014* 


Probabilities 


M2-M3 


.211 


.045 


Ifel 


.018* 


.014* 


Student 


M1-M3 


4 




4 


4 


4 


1 


Newman- 


MrM? 




lillli 


4 


* 




* 


Keuls 


M2-M3 






4 


4 


>4 


1 



* Significant contrast at g< .05. 
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Table 3 



Significant Pairwise Differences for Four Po st-hoc Methods, 
K= 3. Pattern 1-2. n= 100, a= 0 05 





Range 


,350<r 


,575a 


,650a 


,675a 


,700a 




F 


3.06 


8.27 


10.56 


11.39 


12.25 




Pr> F 


.0483 


.0003 


.0000 


.0000 


.0000 




Power 


.59 


.95 


.99 


.99 + 


.99 + 




■7 

CO" 


.017 


.046 


.060 


.065 


.070 
















Procedure 


Contrast 














M 1 -M 3 


* 


♦ 




♦ 




Bonferroni 












• : : 
















Tukey’s 


Mi-^3 


* 








* 


HSD 










• ; .v. 


★ 




W* 








••+T: 




Alpha-Max 


Ml -^3 


.014* 


.000* 




.000* 


.000* 


with Type I 


M.-M2 


.111 


.043 


.022* 


.018* 


.014* 


Probabilities 


3 


.111 


.043 


,022*.. ... 


.018* 


.014* 


Student 




* 




* 


* 


* 


Newman- 


Mi 






* 


* 


* 


Keuls 








♦ 


* 


* 



* Significant contrast at e< -05. 
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best copy available 



Table 4 



Significant Pairwise Differences for Four Post-hoc Methods, 
K= 3. Pattern = 3 . n= 10. a= 0 05 





Range 


1.01a 


1.07<r 


1.11a 


1. 15<r 




F 


3.40 


3.82 


4.11 


4.41 




Pr> F 


.0482 


.0347 


.0277 


.0220 




Power 


.59 


.65 


.68 


.71 




OT 


.138 


.158 


.172 


.185 


Procedure 


Contrast f , 




















MfM 3 










Bonferroni 


\hrv- 3 








Iliill 














Tukey’s HSD 


JVM3 






lllllill | 


* 




M 2 -M 3 












Mi 










Alpha-Max 


M 1 -M 3 


.032 


lllill 


. 020 * 


.016* 


with Type I 


MTM 3 


.032 


1024* 


. 020 * 


.016* 


Probabilities 


MrM? 


1.00 


lllll 


1.00 


1.00 


Student- 


MrM3 


•lliil 


(*) 


(*> 


<*> 


New man - 


M2’M3 


t*> 


<*> 


(*> 


(*) 


Keuls 


Mi-Ms 











* Significant contrast at j)< .05. 

(*) SNK differences tied at r= 3 steps. 
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Table 5 



Significant Pairwise Differences for Four Post-hoc Methods, 
K= 3. Pattern = 3. n= 30. ot = 0.05 





Range 


0.56a 


0.59<r 


0.62a 


0.64 a 




F 


3.14 


3.48 


3.84 


4.10 




Pr> F 


.0484 


.0351 


.0251 


.0200 




Power 


.59 


.64 


.68 


.71 




or 


.045 


.052 


.059 


.064 


Procedure 


Contrast 




ycM 








firfiy 










Bonferroni 


firfiy 












firth 










Tukey’s 


firfi 3 






||||| 


* 


HSD 


fiz'fi) 






* 


* 




firfiy 










Alpha-Max 


firfiy 


.033 


Hitt 


.018* 


.015* 


with Type I 


firfiy 


.033 




.018* 


.015* 


Probabilities ! 


firth 


1.00 




1.00 


1.00 


Student- 


firfiy 


■ 


(*> 


(*) 


(*> 


Newman - 


th-fiy 


| {*) 


(*> 


(*) 


(*) 


Keuls 


firth 











* Significant contrast at j)< .05. 

(*) SNK differences tied at r= 3 steps. 
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Table 6 



Significant Pairwise Differences for Four Post-hoc Methods, 
K= 3, Pattern- 3, n- 100, a- 0.05 





Range 


.305a 


.320(j 


.335(7 


.345(7 




F 


3.10 


3.41 


3.74 


3.97 




Pr> F 


.0465 


.0342 


.0249 


.0199 




Power 


.60 


.64 


.68 


.70 




or 


.014 


.016 


.018 


.019 


Procedure 


Contrast 














MrM 3 








ii* .>£ 


Bonferroni 


Mi"M 3 












Mi "Mi 










Tukey’s 


Mi "Mi 








* 


HSD 


M2-M3 






lllllll 


* 




Mt "Mi 










Alpha-Max 


M1-M3 


.032 


■ii 


.018* 


.015* 


with Type I 


M1-M3 


.032 


J 24 * ■ 


.018* 


.015* 


Probabilities 


Mi -Mi 


1.00 


1.00 


1.00 


1.00 


Student- 


M1-M3 


liliii 


(*) 


(*) 


(*) 


Newman- 


Mi-M3 


llBl 


(*) 


(*) 


(*) 


Keuls 


Mi "Mi 











* Significant contrast at p< -05. 

(*) SNK differences tied at r= 3 steps. 
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Table 7 



Significant Pairwise Differences for Four Post-hoc Methods, 



K— 4, Pattern 1. n = 10, a = 0 05 





Range 


1.35a 


1.90a 


2.36a 


2.44a 


2.50a 




F 


3.04 


6.02 


9.28 


9.92 


10.42 




Pr> F 


.0414 


.0020 


.0001 


.0001 


.0000 




Power 


.67 


.94 


.99 + 


.99 + 


.99 + 




UT 


.133 


.273 


.383 


.401 


.414 


Procedure 
















Contrast 
















Mi-^4 


♦ 




♦ 


★ 


wmmm 

■pi 




Ml-M 3 












Bonferroni 












liiliiil 


















M3-M4 














M2-M3 














MrM 4 


♦ 


* 


* 


II 


★ 




Ml‘M 3 










* 














Tukey’s 


\hrv-* 








1 1 1 


★ 


HSD 












★ 




M3-M4 








lliilli 






M2-M3 








III 






Ml’M 4 


.005* 


.000* 


JQP 


. 000 * 


.000* 


Alpha-Max 




.140 


.041 




. 010 * 


.008* 


with Type I 




.140 


.041 


&\2* 


. 010 * 


.008* 


Probabilities 


M1-M2 


.140 


.041 


ill V 


. 010 * 


.008* 




M3-M4 


.140 


.041 


.0X2* 


. 010 * 


.008* 




M2-M3 


1.00 


1.00 


Jpo jj 


1.00 


1.00 




MrM 4 


* 




★ 


* 


* 














Student 


MrM 3 




(*) 


(*) 


(*) 


(*) 


Newman- 


\hr\L 4 




"{*) 


(*) 


(*) 


(*) 


Keuls 


nrlh 




• (*} 


(*) 


(*) 


(*) 




M 3 ’M 4 




H 


(*) 


(*) 


(*) 




\hr\h 




II®* 









* Significant contrast at p < .05. 

(*) SNK differences tied at r= 3 steps. 
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Table 8 



Significant Pairwise Differences for Four Post-hoc Methods, 



K= 4. Pattern 1. n= 30. 0.05 





Range 


0.75(7 


1.05ct 


1.32<7 


1.35(7 


1.40(7 




F 


2.81 


5.51 


8.71 


9.11 


9.80 




Pr> F 


.0424 


.0014 


.0000 


.0000 


.0000 




Power 


.67 


.93 


.98 


.98 


.99 + 




or 


.043 


.101 


.162 


.169 


.180 


Procedure 


Contrast 






V. * : ." 


V...;, -/..i 
















* 


♦ 


* 


4« 






MrMs 










liiil 


Bonferroni 












||I - 




MrAh 










!|l| 




M 3 -M 4 




























Mr^4 


* 


4« 


* 


! 4* *\. 


* 




McMs 










4> ! 














Tukey’s 


Ihrt* 4 








1111 


* 


HSD 


Mi-Mi 








HHi 


4i 




M 3 -M 4 








ill " 


* 




M 2 -M 3 








liiiii: 






MrM4 


.004* 


. 000 * 




.000* 


. 000 * 


Alpha-Max 


Ml“M3 


.149 


.044 


llllll 


.010* 


.008* 


with Type I 


MrM4 


.149 


.044 


I&12* . 


.010* 


.008* 


Probabilities 




.149 


.044 


■i 


.010* 


.008* 




^3*^4 


.149 


.044 


liiiii 1 


.010* 


.008* 




M2-M3 


1.00 

1 


1.00 


MU 


1.00 


1.00 




MI-M4 


* 




41 


* 


4i 


Student- 


MrM 3 






<*) 


(*) 


(*) 


Newman- 


MrM 4 




|§|f% .. 


<*) 


(*) 


(*) 


Keuls 


Mi -/h 




(*) 


(*) 


(*) 


(*) 




M3-M4 




(*) 


(*) 


(*) | 




MrM 3 





■111 









* Significant contrast at p < *05. 

(*) SNK differences tied at r= 3 steps. 
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Table 9 



Significant Pairwise Differences for Four Post-hoc Methods, 



K= 4, Pattern 1, n = 100, a= 0.05 





Range 


,400a 


.575a 


,710a 


.740a 


.750a 




F 


2.67 


5.51 


8.40 


9.13 


9.38 




Pr> F 


.0475 


.0010 


.0000 


.0000 


.0000 




Power 


.65 


.93 


.99 + 


.99 + 


.99 + 




or 


.012 


.033 


.053 


.057 


.059 






4. 


v... .. . .. : 








Procedure 


Contrast 


v< .O'. 












Mr/*4 


* 


* 


* 


* 






/*r/*3 












Bonferroni 


/*2“/*4 














Mr/*: 










iliil 




/ X 3“/ X 4 














M:“M3 














Ml"M4 


+ 


* 


* 


* • 


★ 




M[-/*3 








l!!!l!l 


* 


Tukey’s 


M2-M4 










* 


HSD 


Pcth 








| 


★ 




M 3 -M 4 








g|g|||| 


★ 




M2~/*3 














Mr/*4 


.005* 


.000* 


, 000 * 


. 000 * 


.000* 


Alpha-Max 


M|-/*3 


.158 


.043 


1UM 


.009* 


.008* 


with Type I 


M2’M4 


.158 


.043 


iiH! 


.009* 


.008* 


Probabilities 


Mr /*2 


.158 


.043 


>. 012 *. • 


.009* 


.008* 




/*3"/*4 


.158 


.043 


';oi2p : 


.009* 


.008* 1 




M 2 -M 3 


1.00 


1.00 


S >00: ' : :l 


1.00 


1.00 




Mr/*4 


★ 




* 


★ 


★ 


Student- 


Mr/*3 




11*1 


(*) 


(*) 


(*) 


Newman- 


M2"/*4 




IWHI 


(*> 


(*) 


(*) 


Keuls 


MrM? 




mill 


(*) 


(*) 


(*) 




/*3‘/*4 






(*) 


(*) 


(*) 




M2’/*3 













* Significant contrast at f> < .05. 

(*) SNK differences tied at r= 3 level. 
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Table 10 



Significant Pairwise Differences for Four Post-hoc Methods^ 
4, Pattern 2, n= 10. a= 0.05 





Range 


1.30<t 


1.60<r 


1.70<t 


1.85(7 


1.90(7 


2.80(7 


3.40(7 


3.70(7 


3.80(7 




F 

Pr> F 


3.13 

.0375 


4.74 

.0069 


5.35 

.0038 


6.34 

.0015 


6.69 

.0011 


14.52 

.0000 


21.41 

.0000 


25.35 

.0000 


26.74 

.0000 




Power 


.68 


.87 


.90 


.95 


.96 


.99 + 


.99 + 


.99 + 


.99 + 






.138 


.219 


.246 


.286 


.299 


.503 


.605 


.646 


.654 


Procedure 


Contrast 






















Bonferrom 


firfiA 

firfii 

firfiA 

firfii 

fiTfi^ 

firfiA 








* 


|§§ 




* 


* 

+ 

* 


Bill 

I»i«! 
II §11111 
|i| 

* :|: 

* * 


Tukey’s 

HSD 


firfiA 

firfi* 

thrfi-A 

firth 

thrfii 

firfiA 


* 


* 




.:1:* ' r\ 

II * • 

:1. * 

;i: : 

. .y. . 

: : v • : ;,v 


* 

* 

* 


* 

* 

* 


* 

* 

* 


. * 

♦ | 

§|g§ HI 

HI!! 


* 

* 

* 

* 

* 

* 


Alpha-Max 
with Type I 
Probabilities 


firfiA 

firth 
thrfi 4 
firth 
firfi* 

firfiA 


.006* 

.061 

.061 

.339 

.339 

.339 


Sfoi*..: 

3122 * 

.241 

:;|41 


.001* 

.016* 

.016* 

.213 

.213 

.213 


.000* 

.009* 

.009* 

.176 

.176 

.176 


.000* 

.008* 

.008* 

.165 

.165 

.165 


.000* 

.000* 

.000* 

.044 

.044 

.044 


J06%r: 

>000* 

Wm- 1 

♦016* 
;oi6* • 


.000* 

.000* 

.000* 

.009* 

.009* 

.009* 


.000*. 

000* 

.000* 

.008* 

.008* 

.008* 


Student- 

Newman- 

Keuls 


firfiA 

firfi 3 
thrfiA 
firth 
thrth 
firfiA 


* 


* 


llilil 

1111:111 

111:!:;-"-":- 


* 

* 

* 


* 


llpll! 

*||I 

Hllllll 

|§||gl 

liill! 


* 

* 

* 

* 

* 

* 


* 

* 

* 

* 

* 

* 


* 



* Significant contrast at g< .05. 
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Table 1 1 



Significant Pairwise Differences for Four Post-hoc Methods, 
K= 4, Pattern 2. n = 30. a — 0.05 





Range 


0.70a 


0.90a 


0.95a 


1.02a 


1.05a 


1.55a 


1.90a 


2.05a 


2.10a 




F 

Pr> F 


2.72 

.0476 


4.50 

.0050 


5.01 

.0026 


5.78 

.0010 


6.13 

.0007 


13.35 

.0000 


20.06 

.0000 


23.35 

.0000 


24.50 

.0000 




Power 


.65 


.87 


.91 


.94 


.95 


.99 + 


.99 + 


.99 + 


.99 + 






.041 


.080 


.091 


.107 


114 


.236 


.323 


.358 


.370 


Procedure 


Contrast 


*.» \.v”y ••• •-'iX 


fill; |i| 




lx 


V **<C.. .W ' : ;$£ 


lillS 


mm 


p 


em l 


Bonferroni 


fi\-fi* 

firth 

fil'fii 

firfi 2 
firfi* 
fi*~fi* 








★ 


iiiii! 

llllillllll 


+ 




+ 

+ 


J 

1 1 . 
* • : ' 


Tukey’s HSD 


fiCfi* 

firfi* 

fiffi* 

firth 

firfi* 

firfi* 


+ 


+ 


+ 


V"* 

ij jii( ! 


+ 

+ 

* 

j 


+ 

* 

+ 


★ 

* 

I 


lilllll! 
|g|| • «*:• 

Bill 


+ 

+ 

+ 

+ 

+ 

* 


Alpha-Max 
with Type I 
Probabilities 


firfi* 

firfi* 

firfi* 

fi\'th 

fiTfi* 

firfi* 


.008* 

.073 

.073 

.368 

.368 

.368 




.000* 

.016* 

.016* 

.223 

.223 

.223 


.000* 

.010* 

.010* 

.190 

.190 

.190 


.000* 

.008* 

.008* 

.178 

.178 

.178 


.000* 

.000* 

.000* 

.048 

.048 

.048 


>000*:?. 

,000*: 

Ji!*!! 

■m 

ittH 


.000* 

.000* 

.000* 

.009* 

.009* 

.009* 


.000* 

.000* 

.000* 

.008* 

.008* 

.008* 


Student- 

Newman- 

Keuls 


^r^4 

^r^3 

thrfi* 
firth 
thrfi 3 
fir fit 


+ 


★ 


i * 

: i 

•X&r$&>X\vX;X;X; 

--v> j 


★ 

★ 

★ 


+ 

+ 

+ 


■Hi 

l|lp|l§§ 

llllllll 


+ 

★ 

+ 

★ 

+ 

* 


* 

* 

+ 

+ 

+ 

+ 


+ 



* Significant contrast at p < .05. 
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Table 12 



Significant Pairwise Differences for Four Post-hoc Methods^ 



K= 4, Pattern 2. n= 100, q= 0.05 





Range 


0.40a 


0.50a 


0.52a 


0.55a 


0.60a 


0.85a 


1.05a 


1.10a 


1. 15a 




F 


2.96 


4.63 


5.01 


5.60 


6.67 


13.38 


20.42 


22.41 


24.49 




Pr> F 


.0320 


.0034 


.0020 


.0009 


.0002 


.0000 


.0000 


.0000 


.0000 




Power 


.70 


.87 


.90 


.94 


.96 


.99 + 


.99 + 


.99 + 


.99 + 




op" 


.019 


.035 


.038 


.044 


.053 


.110 


.162 


.176 


.190 


Procedure 


Contrast 










































Ml-^4 


* 


* 


4* 


* 






* 


* 


lllillll 
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Table 13 



Significant Pairwise Differences for Four Post-hoc Methods, 



K= 4, Pattern 3. n= 10. a= 0 05 





Range 


.95a 


1.20<r 


1.22a 


1.25a 




F 


3.01 


4.80 


4.96 


5.21 




Pr> F 


.0428 


.0065 


.0055 


.0043 




Power 


.66 


.87 


.88 


.89 




a r 


.131 


.222 


.229 


.240 


Procedure 


Contrast 


Si 


Mi 


ipgpp 


















MrMs 




















••• 


Bonferroni 


M2-M3 












M:"M 4 












MrMi 












M3-M4 












Ml“M 3 






‘ * 


★ 




Ml"M 4 




1 


ill *••• . " 


★ 


Tukey’s 


M2-M3 








4* 


HSD 


M2-M4 








* 




Mi "M2 












M3-M4 












M1-M3 


.041 


liWii! 


.010* 


.008* 


Alpha-Max 


MrM 4 


.041 


in* 


.010* 


.008* 


with Type I 


M2-M3 


.041 


life; 


.010* 


.008* 


Probabilities 


MrM 4 


.041 


iilli 


.010* 


.008* 




Mi -M2 


1.00 


iSil 


1.00 


1.00 




M3-M4 


1.00 


iliili 


1.00 


1.00 




M1-M3 


■HU 


(*) 


(*) 


(*) 


Student- 


M1-M4 


■il 


<*) 


n 


(*) 


Newraan- 


M2-M3 


■1 


(*) 


n 


0 


Keuls 


M2-M4 


— 


(*) 


n 


0 




Mi -M2 


Illlli 










M3-M4 












* Significant contrast at g<. 05. 

(*) SNK differences tied at r= 4 steps. 
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Table 14 



Significant Pairwise Differences for Four Post-hoc Methods, 
K= 4, Pattern 3. n= 30, a= 0.05 
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* Significant contrast at j>< .05. 

(*) SNK differences tied at r= 4 steps. 
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Table 15 



Significant Pairwise Differences for Four Post-hoc Methods, 
K— 4, Pattern 3, n— 100, a — 0.05 





Range 
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* Significant contrast at g< .05. 

(*) SNK differences tied at r= 4 steps. 
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